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Amendment dated August 10, 2009 

Responsive to Final Office Action dated May 8, 2009 

REMARKS 

Claims 1,9, 17 and 25 have been amended. No new matter is presented by these 
amendments. 

Claims 1-4, 6-1 1, 13-15, and 17-40 are pending in the application. 

Applicant submits this amendment in conjunction with the Request for Continued 
Examination being concurrently filed herewith. Notwithstanding Applicant's belief that 
the claims as previously presented were patentable over the prior art, Applicant has 
amended the claims in order to expedite allowance of the present application. In 
particular, Applicants have amended the independent claims to recite the feature of 
tracking an object within the foreground region, the tracking being enhanced by the 
applied adjusted image capture device parameter. 

Rejections Under 35 U.S.C. § 103 

In the Final Office Action dated May 8, 2009, claims 1, 3-4, 6-11,13-15, 17-32, 
34-35, 37 and 39-40 were rejected under 35 USC § 103(a) as being unpatentable over 
Gvili et al CGviir) (Depth Keying, SPIE Vol. 5006(2003)) in view of Luo et al. ("Lwo") 
(US 2002/0093670), and further in view of Clarke et al ("Clarke") (US 4,591,842). 
However, as discussed in further detail below, the combined teachings of the cited prior 
art references would not have rendered the subject matter of Applicants' claims obvious 
to one having ordinary skill in the art. 

I. The Gvili reference teaches extraction of foreground objects for 
recomposition on a new background. 



Amendment 



12 



Atty. Docket No. SONYP03 1 



Application No. 10/758,817 

Amendment dated August 10, 2009 

Responsive to Final Office Action dated May 8, 2009 

Gvili teaches a method of video keying which utilizes a depth video camera. As 
is known in the art, video keying involves determining an object of interest within a video 
sequence based on a key, and excluding or ignoring all other objects within the video 
sequence. Gvili refers to objects of interest as the foreground, while objects that are to be 
excluded are referred to as the background, regardless of their actual position in the 
scene. Gvili, page 564. Gvili 's technique is drawn to a method of mixing video from 
different sources, wherein depth information is applied as the key by which objects of 
interest (the foreground) are determined. The identified objects of interest are thus 
preserved in a given video sequence, while the remainder of the video sequence (the 
background) is overlayed or replaced with video from a separate source. 

Additionally, Gvili teaches a method of color spill correction to enable recovery 
of the pure color of foreground objects. The color spill correction is performed before the 
foreground objects are composed on a new background, to prevent residues of old 
background colors from being visible in the composite video. Gvili, page 571. 

In contrast to Gvili, Applicant's independent claims are directed to the 
independent adjustment of image capture device parameters of brightness, exposure, and 
gain, as applied to both foreground and background regions of a scene, based on bit 
values of a depth mask. Thus while Gvili teaches extraction of objects of interest for 
recomposition on a new background, Applicant's claimed invention is drawn to 
adjustment of image capture device parameters so that both the foreground and 
background regions of an original image of a scene are more aesthetically pleasing and 
conducive to tracking. 
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II. The Luo reference teaches a post-processing subject detection algorithm. 

As noted by the Office, Gvili does not teach the adjusting of image capture device 
parameters according to bit values of the depth mask, wherein the image capture device 
parameters are selected from one of brightness, exposure or gain. For this feature of 
Applicant's claimed invention, the Office cites the Luo reference. 

Luo teaches a photofinishing method for producing modified versions of original 
pictures according to an analysis of the subject content of the photographic image. In 
particular, Luo teaches a main subject detection algorithm which evaluates regions of 
homogeneous properties (such as color and texture) for saliency in terms of structural 
features such as centrality and borderness, and semantic features such as the presence of 
skin tones. The main subject detection algorithm thus produces a list of segmented 
regions ranked in descending order of the likelihood that each is a main subject (a 
gradient of belief values), which may be converted to a belief map. A probable main 
subject may be emphasized based on the belief map. Luo t paragraphs [0041]-[0056]. 

However, in contrast to Applicant's claimed invention, which is directed to 
adjustment of image capture device parameters, Luo 's disclosure is directed to post- 
processing of still photos separate and apart from the image capture device which took 
the photos in the first instance. Moreover, Applicant's adjustment of image capture 
device parameters is according to bit values of a depth mask, whereas the main subject 
emphasis taught by Luo is not related to depth, but rather based on belief values which 
are the result of a detection algorithm applied to a photographic image. 
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III. One skilled in the art would not have been motivated to combine the 
teachings of Gvili and Luo. 

In contrast to GvilVs video keying process, which teaches identification of objects 
of interest definitively based on depth information, Luo teaches identification of regions 
in still photos with associated belief values that merely represent a likelihood that each is 
a main subject based on features such as location within the photo. Luo 's belief values 
have nothing to do with depth, while Gvili 's objects of interest are entirely based on 
depth. 

Moreover, because Luo 's subject detection algorithm determines probabilities that 
a region might constitute a main subject, Luo teaches that applying subject emphasis in a 
gradual manner according to the gradient of belief values can mitigate the sense that the 
main subject was inaccurately identified, and help conceal the artifacts introduced by an 
inaccurate determination of main subject and the background. Luo, paragraph [0056]. 
Similarly, the Office notes that at paragraph [0018], Luo teaches that artifacts due to 
errors in content analysis are not noticeable and objectionable. However, because Gvili 
defines objects of interest based solely on depth, there is no question as to the accuracy or 
inaccuracy of determining objects of interest, as the objects of interest are by definition 
those having a certain depth. Gvili 's method does not require "content analysis" as 
taught by Luo to determine objects of interest, as they are simply determined based on 
depth information. 

Thus, Applicant submits that one skilled in the art would not have been motivated 
to combine the teachings of Luo with those of Gvili. Gvili 's method relates to video 
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keying based on depth information, whereas Luo teaches probabilistic detection of a main 
subject in a photo based on factors excluding depth information. 

IV. The Clarke reference teaches foreground/background colors which are 
unrelated to Applicant's claimed invention. 

As noted by the Office, neither Gvili nor Luo teach the application of adjusted 
image capture device parameters so that the brightness, exposure, or gain of the captured 
image frames is adjustable independently for both of the objects within the foreground 
region and the objects within the background region. For this feature of Applicant's 
claimed invention, the Office relies upon the Clarke reference. 

Clarke teaches a raster graphic system in which a set of adjacent pixels are 
simultaneously addressed to reduce memory consumption. A background/foreground 
control bit determines if a pixel being scanned is to have a foreground or a background 
color. Clarke, Col. 3-4. Thus, while Clarke teaches foreground and background colors, 
they are merely different colors applied to pixels in a raster graphic system, and do not 
relate to Applicant's claimed foreground and background regions. Indeed, an object 
within either of the foreground or background regions of Applicant's claimed invention 
could contain both "foreground" and "background" colors as defined by Clarke. As 
such, the Clarke reference is not relevant to Applicant's claimed invention. 

In sum, Applicant's independent claims 1,9, 17, and 25 are patentable over the 
combined teachings of Gvili, Luo, and Clarke. Likewise, the dependent claims are 
patentable for at least the same reasons as the independent claims. 
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V. The Luo reference does not teach tracking of objects within a foreground 
region. 

Claims 34, 35, 37, and 39 relate to tracking of objects within the foreground 
region. However, the Office rejected the subject matter of these claims over the 
teachings of Luo. Applicant submits that Luo does not relate to tracking of objects 
because Luo merely teaches a process for modifying still photographs. Moreover, Luo 's 
subject emphasis is based on a belief map, as discussed above, which does not relate to 
the Applicant's claimed foreground and background objects, which are identified based 
on a depth mask. In fact, because Luo teaches that the location of an object within a 
photographic image affects its saliency for subject detection purposes, movement of an 
object to a different location within a photographic image would change its belief value, 
and could result in a different object being emphasized. This would not facilitate 
tracking of the original object, but would in fact make tracking more difficult. 

VI. The Tuomi and Podoleanu references do not cure the deficiencies of Gvili, 
Luo, and Clarke. 

Claim 2 was rejected under 35 U.S.C. 103(a) as being unpatentable over Gvili, in 
view of Luo and Clarke, and further in view of Tuomi et ah ("Tuomi") (US 7,061,507). 
Claims 33, 36, and 38 were rejected under 35 U.S.C. 103(a) as being unpatentable over 
Gvili in view of Luo and Clarke, and further in view of Podoleanu et ah ^Podoleanu") 
(US 6,769,769). Claims 2, 33, 36, and 38 depend from independent claims 1,17, and 25 
The teachings of Tuomi and Podoleanu do not cure the deficiencies of the prior art with 
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respect to the independent claims as discussed above. Therefore, claims 2, 33, 36, and 38 
are patentable for at least the same reasons as their corresponding independent claims. 

VII. The prior art references do not disclose Applicants' claimed enhanced 
tracking of foreground objects. 

Applicants have amended independent claims 1,9, 17, and 25 to recite the feature 
of tracking an object within the foreground region, the tracking being enhanced by the 
applied adjusted image capture device parameter. Support for this amendment may be 
found in Applicants' specification at, for example, paragraphs, [0033] and [0041]. The 
prior art references do not disclose enhanced tracking of an object as claimed. And as 
discussed above, the Luo reference's teachings would actually be detrimental to tracking 
of objects. 

Conclusion 

In light of the amendments to the claims, and the foregoing remarks highlighting 
the differences between the claimed invention and the teachings of the cited prior art, the 
Applicant respectfully submits that the pending claims are in condition for allowance. A 
notice of allowance is respectfully requested. 

In the event a telephone conversation would expedite the prosecution of this 
application, the Examiner may reach the undersigned at (408) 774-6903. If any fees are 
due in connection with the filing of this paper, then the Commissioner is authorized to 
charge such fees to Deposit Account No. 50-0805 (Order No. SONYP03 1). A copy of 
the transmittal is enclosed for this purpose. 
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Respectfully submitted, 

MARTINE PENILLA & GENCARELLA, LLP 

/Albert S. Penilla/ 

Albert S. Penilla, Esq. 
Registration No. 39,487 

710 Lakeway Drive, Suite 200 
Sunnyvale, California 94085 
Telephone: (408) 774-6903 
Customer No. 25920 
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